Transferable Pedestrian Motion Prediction Models at Intersections

One desirable capability of autonomous cars is to accurately predict the pedestrian motion near intersections for safe and efficient trajectory planning. We are interested in developing transfer learning algorithms that can be trained on the pedestrian trajectories collected at one intersection and yet still provide accurate predictions of the trajectories at another, previously unseen intersection. We first discussed the feature selection for transferable pedestrian motion models in general. Following this discussion, we developed one transferable pedestrian motion prediction algorithm based on Inverse Reinforcement Learning (IRL) that infers pedestrian intentions and predicts future trajectories based on observed trajectory. We evaluated our algorithm on a dataset collected at two intersections, trained at one intersection and tested at the other intersection. We used the accuracy of augmented semi-nonnegative sparse coding (ASNSC), trained and tested at the same intersection as a baseline. The result shows that the proposed algorithm improves the baseline accuracy by 40% in the non-transfer task, and 16% in the transfer task

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches

Multipath TCP (MPTCP) has been widely used as an efficient way for communication in many applications. Data centers, smartphones, and network operators use MPTCP to balance the traffic in a network efficiently. MPTCP is an extension of TCP (Transmission Control Protocol), which provides multiple paths, leading to higher throughput and low latency. Although MPTCP has shown better performance than TCP in many applications, it has its own challenges. The network can become congested due to heavy traffic in the multiple paths (subflows) if the subflow rates are not determined correctly. Moreover, communication latency can occur if the packets are not scheduled correctly between the subflows. This paper reviews techniques to solve the above-mentioned problems based on two main approaches; non data-driven (classical) and data-driven (Machine Learning) approaches. This paper compares these two approaches and highlights their strengths and weaknesses with a view to motivating future researchers in this exciting area of machine learning for communications. This paper also provides details on the simulation of MPTCP and its implementations in real environments.Comment: 13 pages, 7 figure

Robustness Analysis of Neural Networks via Efficient Partitioning with Applications in Control Systems

Neural networks (NNs) are now routinely implemented on systems that must operate in uncertain environments, but the tools for formally analyzing how this uncertainty propagates to NN outputs are not yet commonplace. Computing tight bounds on NN output sets (given an input set) provides a measure of confidence associated with the NN decisions and is essential to deploy NNs on safety-critical systems. Recent works approximate the propagation of sets through nonlinear activations or partition the uncertainty set to provide a guaranteed outer bound on the set of possible NN outputs. However, the bound looseness causes excessive conservatism and/or the computation is too slow for online analysis. This paper unifies propagation and partition approaches to provide a family of robustness analysis algorithms that give tighter bounds than existing works for the same amount of computation time (or reduced computational effort for a desired accuracy level). Moreover, we provide new partitioning techniques that are aware of their current bound estimates and desired boundary shape (e.g., lower bounds, weighted $\ell_\infty$-ball, convex hull), leading to further improvements in the computation-tightness tradeoff. The paper demonstrates the tighter bounds and reduced conservatism of the proposed robustness analysis framework with examples from model-free RL and forward kinematics learning

Evaluation of Lipid Profile and PCSK9 Serum Levels in Parkinson’s Patients in Comparison with Healthy Subjects

Introduction Up to now, limited and contradictory results have been published on the role of prognostic values of lipid profile molecules including: HDL (High Density Lipoprotein), LDL (Low Density Lipoprotein), TG (Triglyceride), Total Cholesterol and PCSK9 (Proprotein Convertase SubtilisinKexin type 9) molecule in occurrence and development of Parkinson’s disease (PD). The aim of this study was to investigate the role of lipid profile and PCSK9 in patients with PD and to compar it with healthy individuals. Methods and Results In the present case-evidence study, 32 individuals diagnosed with PD were compared with 32 healthy individuals. After receiving the  participant's consent forms, 5 ml blood was taken from vein and the level of HDL(High -Density Lipoprotein), LDL (Low-Density Lipoprotein), TG (Triglyceride),Total Cholesterol and PCSK9 in the blood samples were measured. The Elisa method was used for measuring PCSK9 level in blood serum. Data were analyzed using SPSS17 software. The P values smaller than 0.05 were considered significant. The mean age of participants in the PD and control group was 56.9±8.8 and 53.7±10.1 years respectively (P>0.05). Twenty seven individuals (87.1%) and 13 individuals (41.9%) in the PD group and control group were men, respectively. The remaining participants were women (P=0.000). LDL level (84.2±24.9 ml/dl vs. 105.5±16.8, P=0.000), HDL (45.5±8.7 ml/dl vs. 51.1±9.5 ml/dl, P=0.000), total cholesterol (155.3±31.2 ml/dl vs. 192.8±32.5 ml/dl P=0.000) were lower and TG level was higher in the PD group (133.3±79.3 ml/dl vs. 131.2±58.6 ml/dl, P=0.9) compared with the control group. PCSK9 level was higher in the PDgroup, but no significant difference was found (141.6±70 vs. 129.7±51 ng/ml, P=0.5). ConclusionsOur findings showed that individuals with PD have lower level of HDL, LDL and total cholesterol compared with the control group, but PCSK9 levels were same in both groups

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

A fundamental challenge in multiagent reinforcement learning is to learn beneficial behaviors in a shared environment with other simultaneously learning agents. In particular, each agent perceives the environment as effectively non-stationary due to the changing policies of other agents. Moreover, each agent is itself constantly learning, leading to natural non-stationarity in the distribution of experiences encountered. In this paper, we propose a novel meta-multiagent policy gradient theorem that directly accounts for the non-stationary policy dynamics inherent to multiagent learning settings. This is achieved by modeling our gradient updates to consider both an agent's own non-stationary policy dynamics and the non-stationary policy dynamics of other agents in the environment. We show that our theoretically grounded approach provides a general solution to the multiagent learning problem, which inherently comprises all key aspects of previous state of the art approaches on this topic. We test our method on a diverse suite of multiagent benchmarks and demonstrate a more efficient ability to adapt to new agents as they learn than baseline methods across the full spectrum of mixed incentive, competitive, and cooperative domains.Comment: Accepted to ICML 2021. Code at https://github.com/dkkim93/meta-mapg and Videos at https://sites.google.com/view/meta-mapg/hom

Modeling of cerebellar transcranial electrical stimulation effects on hand tremor in Parkinson’s disease

IntroductionParkinson’s disease (PD) is a neurodegenerative disorder with different motor and neurocognitive symptoms. Tremor is a well-known symptom of this disease. Increasing evidence suggested that the cerebellum may substantially contribute to tremors as a clinical symptom of PD. However, the theoretical foundations behind these observations are not yet fully understood.MethodsIn this study, a computational model is proposed to consider the role of the cerebellum and to show the effectiveness of cerebellar transcranial alternating current stimulation (tACS) on the rest tremor in participants with PD. The proposed model consists of the cortex, cerebellum, spinal circuit-muscular system (SC-MS), and basal ganglia blocks as the most critical parts of the brain, which are involved in generating rest tremors. The cortex, cerebellum, and SC-MS blocks were modeled using Van der Pol oscillators that interacted through synchronization procedures. Basal ganglia are considered as a regulator of the coupling weights defined between oscillators. In order to evaluate the global behavior of the model, we applied tACS on the cerebellum of fifteen PD patients for 15 min at each patient’s peak frequency of their rest tremors. A tri-axial accelerometer recorded rest tremors before, during, and after the tACS.Results and DiscussionThe simulation of the model provides a suggestion for the possible role of the cerebellum on rest tremors and how cerebellar tACS can affect these tremors. Results of human experiments also showed that the online and offline effects of cerebellar tACS could lead to the reduction of rest tremors significantly by about %76 and %68, respectively. Our findings suggest that the cerebellar tACS could serve as a reliable, therapeutic technique to suppress the PD tremor

Collective Transport of an Unknown Object by Multi Robots with Limited Sensing

This thesis presents a fully distributed approach to retrieve a large object from an unknown environment. The object is assumed to be located in an environment without GPS or Internet infrastructure. The object is too heavy to transport by one robot. The collective transport problem is broken into five major steps: 1) Exploring the unknown environment and finding the object. 2) Grasping the object. 3) Characterizing the object. 4) Planning a path to the desired location. 5) Transporting the object to the desired location. This thesis presents efficient distributed algorithms for robots with limited sensing to accomplish steps three to five. Object characterization includes centroid estimation and object dimension estimation. Two algorithms are developed for centroid estimation. In the first algorithm, each robot uses a communication tree to compute the sum of its children's positions. The second algorithm is based on pipelined consensus, which is an extension of pairwise gossip-based consensus. Two algorithms are presented to estimate object's dimensions. The first one is a distributed principal component analysis algorithm, and the second one is the distributed version of rotating calipers algorithm. A distributed path planning algorithm is presented. Robots have already been scattered across the terrain and collectively sample the obstacles in the environment. Robots use this sampling along with the estimated dimensions of the object, from above, to construct a configuration space of robots and the object. A variant of the distributed Bellman-Ford algorithm is then used to construct a shortest-path tree. A path navigation algorithm is presented to map each path segments to a distributed motion controller that can command the robots to transport the object. Four distributed motion controllers are designed including: rotation around a pivot robot, rotation in-place around an estimated centroid of the object, translation, and a combined motion of rotation and translation. Finally, a distributed recovery algorithm is presented to recover the robots efficiently and safely after collective transport. This recovery method uses k-redundant maximum-leaf spanning trees that guarantee connectivity during the recovery. All algorithms are verified through simulation as well as hardware experiments. The results are promising, and the algorithms successfully transport convex or concave objects in simulation and hardware experiments. After robots transport the object, robots are successfully recovered at home location by using the recovery algorithm. All algorithms discussed in this thesis are fully distributed, efficient, and robust to object shape and network population changes